Hearing assistance devices with motion sickness prevention and mitigation features

ABSTRACT

Embodiments herein relate to devices and related systems and methods for motion sickness prevention and mitigation. In an embodiment, a method of preventing or mitigating motion sickness in a subject is included, the method tracking motion of the subject using a first motion sensor; estimating a vestibular system input based on tracked motion of the subject; tracking head position of the subject using the first motion sensor; estimating a visual system input based on tracked head position of the subject; estimating consistency between the vestibular system input and the visual system input; and initiating a responsive measure if the estimated consistency crosses a threshold value. Other embodiments are also included herein.

This application is a continuation of U.S. patent application Ser. No.16/803,639, filed Feb. 27, 2020, which claims the benefit of U.S.Provisional Application No. 62/811,367, filed Feb. 27, 2019, the contentof which is herein incorporated by reference in its entirety.

FIELD

Embodiments herein relate to devices and related systems and methods formotion sickness prevention and mitigation. More specifically,embodiments herein relate to hearing assistance devices with motionsickness prevention and mitigation features.

BACKGROUND

The brain receives various inputs that can indicate motion. Motion canbe sensed primarily by the body's vestibular system and visual system,and to a lesser extent by other systems such as the somatosensory andproprioceptive systems. The brain synthesizes received inputs to provideappropriate sensations with respect to movement. For example, when aperson is walking, the various inputs to the brain are typically highlyconsistent with one another and result in an appropriate sensation ofmovement.

However, in some individuals or situations, received vestibular andvisual inputs are not consistent with one another. These inconsistentinputs can lead to sensations and symptoms associated with motionsickness. Inconsistent inputs can be caused by particular activities,disease states, impairments, or the like.

Inconsistent vestibular and visual inputs received by the brain cancause motion sickness in some people. Motion sickness may result innausea, vomiting, dizziness, malaise, headache, pallor, palpitations,anxiety, fatigue, difficulty maintaining balance, unsteadiness, impairedcognitive function and the like.

One example of an activity leading to inconsistent vestibular and visualinputs is an individual riding in a car with their gaze down on astationary object (stationary in a frame relative to the vehicle) suchas a book. In such a case, their vestibular system will still provide aninput to the brain that is consistent with motion. However, with theirgaze down on the book, their visual system will not be providing aninput that indicates motion.

SUMMARY

Embodiments herein relate to devices and related systems and methods formotion sickness prevention and mitigation. In an embodiment, a method ofpreventing or mitigating motion sickness in a subject is included. Themethod can include tracking motion of the subject using a first motionsensor, estimating a vestibular system input based on tracked motion ofthe subject, tracking head position of the subject using the firstmotion sensor, estimating a visual system input based on tracked headposition of the subject, estimating consistency between the vestibularsystem input and the visual system input, and initiating a responsivemeasure if the estimated consistency crosses a threshold value.

In an embodiment, a method of preventing or mitigating motion sicknessin a subject is included herein, the method including tracking motion ofthe subject using a first motion sensor associated with a hearingassistance device, tracking head position of the subject using the firstmotion sensor, capturing ambient sounds with the hearing assistancedevice, classifying the surroundings based on the tracked motion,tracked head position, and ambient sound as one of vehicular,stationary, and non-stationary, and initiating a responsive measurebased on the classification of the surroundings.

In an embodiment, a motion sickness prevention system is included havinga first control circuit and a first motion sensor in electricalcommunication with the first control circuit. The first motion sensorcan be disposed in a fixed position relative to a head of a subjectwearing the hearing assistance device. The system can further include afirst microphone in electrical communication with the first controlcircuit, a first electroacoustic transducer for generating sound inelectrical communication with the first control circuit, and a firstpower supply circuit in electrical communication with the first controlcircuit. The first control circuit can be configured to track motion ofthe subject using the first motion sensor, estimate vestibular systeminput based on tracked motion of the subject, track head position of thesubject using the first motion sensor, and estimate visual system inputbased on tracked head position of the subject.

In an embodiment, a motion sickness prevention system is included havinga first control circuit and a first motion sensor in electricalcommunication with the first control circuit. The first motion sensorcan be disposed in a fixed position relative to a head of a subjectwearing the hearing assistance device. The system can further include afirst microphone in electrical communication with the first controlcircuit, a first electroacoustic transducer for generating sound inelectrical communication with the first control circuit, and a firstpower supply circuit in electrical communication with the first controlcircuit. The first control circuit can be configured to track motion ofa subject using the motion sensor, track head position of a subjectusing the motion sensor, sense vibrations with a vibration sensor, senseambient sounds with the microphone, and sense magnetic fields with amagnetic field sensor. The first control circuit can be configured toclassify the surroundings based on the tracked motion, tracked headposition, sensed vibrations, sensed ambient sound, and sensed magneticfields, as one of as one of vehicular, stationary, and non-stationary.The first control circuit can be further configured to initiate aresponsive measure based on the classification of the surroundings.

This summary is an overview of some of the teachings of the presentapplication and is not intended to be an exclusive or exhaustivetreatment of the present subject matter. Further details are found inthe detailed description and appended claims. Other aspects will beapparent to persons skilled in the art upon reading and understandingthe following detailed description and viewing the drawings that form apart thereof, each of which is not to be taken in a limiting sense. Thescope herein is defined by the appended claims and their legalequivalents.

BRIEF DESCRIPTION OF THE FIGURES

Aspects may be more completely understood in connection with thefollowing figures (FIGS.), in which:

FIG. 1 is a partial cross-sectional view of ear anatomy.

FIG. 2 is a schematic view of a hearing assistance device in accordancewith various embodiments herein.

FIG. 3 is a schematic view of various components of a hearing assistancedevice in accordance with various embodiments herein.

FIG. 4 is a schematic view of a hearing assistance device disposedwithin the ear of a subject in accordance with various embodimentsherein.

FIG. 5 is a schematic side view of a subject wearing a hearingassistance device in accordance with various embodiments herein.

FIG. 6 is a schematic top view of a subject wearing a hearing assistancedevice in accordance with various embodiments herein.

FIG. 7 is a schematic side view of a subject riding in a car inaccordance with various embodiments herein.

FIG. 8 is a schematic top view of a subject riding in a car inaccordance with various embodiments herein.

FIG. 9 is a schematic side view of a subject riding in a boat inaccordance with various embodiments herein.

FIG. 10 is a schematic view of an external visual display device andelements of a display screen thereof in accordance with variousembodiments herein.

While embodiments are susceptible to various modifications andalternative forms, specifics thereof have been shown by way of exampleand drawings and will be described in detail. It should be understood,however, that the scope herein is not limited to the particular aspectsdescribed. On the contrary, the intention is to cover modifications,equivalents, and alternatives falling within the spirit and scopeherein.

DETAILED DESCRIPTION

As described above, inconsistent vestibular system and visual systeminputs can lead to motion sickness. In accordance with variousembodiments herein, systems and methods are provided by preventingand/or mitigating motion sickness in a subject.

In various embodiments herein, a system such as a hearing assistancedevice can track the motion or movement of a subject using a motionsensor. In some embodiments, the system can estimate vestibular systeminput based on the tracked motion of the subject. The head position ofthe subject can also be tracked. In some embodiments, the system canestimate visual system input based on the tracked head position. Thesystem can estimate consistency between or otherwise compare thevestibular system inputs and the visual system inputs. In variousembodiments, the system can also initiate responsive measures if theestimated consistency crosses a threshold value.

The term “motion sickness” as used herein shall include reference to seasickness, air sickness, travel sickness, space sickness, screen sicknessand virtual reality stimulation sickness, unless the context dictatesotherwise.

The term “hearing assistance device” as used herein shall refer todevices that can aid a person with impaired hearing. The term “hearingassistance device” shall also refer to devices that can produceoptimized or processed sound for persons with normal hearing. The term“hearing assistance device” shall also refer to devices worn on or aboutthe ear, and that may produce sound or perform other useful functionsfor the user. Hearing assistance devices herein can include hearables(e.g., wearable earphones, headphones, earbuds, virtual realityheadsets), hearing aids (e.g., hearing instruments), cochlear implants,and bone-conduction devices, for example. Hearing assistance devicesinclude, but are not limited to, behind-the-ear (BTE), in-the ear (ITE),in-the-canal (ITC), invisible-in-canal (IIC), receiver-in-canal (RIC),receiver in-the-ear (RITE) or completely-in-the-canal (CIC) type hearingassistance devices or some combination of the above.

Referring now to FIG. 1, a partial cross-sectional view of ear anatomy100 is shown. The three parts of the ear anatomy 100 are the outer ear102, the middle ear 104 and the inner ear 106. The inner ear 106includes the cochlea 108, vestibule 117, the semicircular canals 118,and cranial nerve VIII (the vestibulocochlear/auditory nerve) 120. Theouter ear 102 includes the pinna 110, ear canal 112, and the tympanicmembrane 114 (or eardrum). The middle ear 104 includes the tympaniccavity 115 and auditory bones 116 (malleus, incus, stapes). Thepharyngotympanic tube 122 (i.e., the eustachian tube) is an air pathwayin communication with the middle-ear space and nasopharynx. Theeustachian tube helps to control pressure within the middle ear space,generally making it equal with, e.g., ambient air pressure.

Sound waves enter the ear canal 112 and make the tympanic membrane 114vibrate. This action oscillates the tiny chain of auditory bones 116(ossicles—malleus, incus, stapes) in the middle ear 104. The medial bonein this chain contacts the oval window membrane of the cochlea 108 andtransfers the oscillations to the fluid in the cochlea 108. The threeossicles (i.e., ossicular chain) create a lever action and help toovercome the air-to-fluid impedance miss-match along the air-conductionauditory pathway. The fluid movement eventually results in a neuralresponse along the auditory nerve 120.

Hearing assistance devices, such as hearing aids and hearables (e.g.,wearable earphones), can include an enclosure, such as a housing orshell, within which internal components are disposed. Components of ahearing assistance device herein can include a control circuit, digitalsignal processor (DSP), memory (such as non-volatile memory), powermanagement circuitry, a data communications bus, one or morecommunication devices (e.g., a radio, a near-field magnetic inductiondevice), one or more antennas, one or more microphones, areceiver/speaker, and various sensors as described in greater detailbelow. More advanced hearing assistance devices can incorporate along-range communication device, such as a BLUETOOTH® transceiver orother type of radio frequency (RF) transceiver.

Referring now to FIG. 2, a schematic view of a hearing assistance device200 is shown in accordance with various embodiments herein. The hearingassistance device 200 can include a hearing device housing 202. Thehearing device housing 202 can define a battery compartment 210 intowhich a battery can be disposed to provide power to the device. Thehearing assistance device 200 can also include a receiver 206 adjacentto an earbud 208. The receiver 206 an include a component that convertselectrical impulses into sound, such as an electroacoustic transducer,speaker, or loud speaker. A cable 204 or connecting wire can include oneor more electrical conductors and provide electrical communicationbetween components inside of the hearing device housing 202 andcomponents inside of the receiver 206.

The hearing assistance device 200 shown in FIG. 2 is a receiver-in-canaltype device and thus the receiver is designed to be placed within theear canal. However, it will be appreciated that many different formfactors for hearing assistance devices are contemplated herein. As such,hearing assistance devices herein can include, but are not limited to,behind-the-ear (BTE), in-the ear (ITE), in-the-canal (ITC),invisible-in-canal (IIC), receiver-in-canal (RIC), receiver in-the-ear(RITE), completely-in-the-canal (CIC) type hearing assistance devices,cochlear implants, and osseo-integrated/bone-conduction type devices.

Hearing assistance devices of the present disclosure can incorporate anantenna arrangement coupled to a high-frequency radio, such as a 2.4 GHzradio. The radio can conform to an IEEE 802.11 (e.g., WIFI) orBLUETOOTH® (e.g., BLE, BLUETOOTH® 4. 2 or 5.0) specification, forexample. It is understood that hearing assistance devices of the presentdisclosure can employ other radios, such as a 900 MHz radio. Hearingassistance devices of the present disclosure can be configured toreceive streaming audio (e.g., digital audio data or files) from anelectronic or digital source. Representative electronic/digital sources(also referred to herein as accessory devices) include an assistivelistening system, a TV streamer, a radio, a smartphone, a cellphone/entertainment device (CPED), remote control, or other electronicdevice that serves as a source of digital audio data, settingconfigurations, commands, or files.

Referring now to FIG. 3, a schematic block diagram is shown with variouscomponents of a hearing assistance device in accordance with variousembodiments. The block diagram of FIG. 3 represents a generic hearingassistance device for purposes of illustration. The hearing assistancedevice 200 shown in FIG. 3 includes several components electricallyconnected to a flexible mother circuit 318 (e.g., flexible mother board)which is disposed within housing 300. A power supply circuit 304 caninclude a battery and can be electrically connected to the flexiblemother circuit 318 and provides power to the various components of thehearing assistance device 200. In other examples, components of ahearing assistance device 200 may draw electrical power from anothertype of power source. One or more microphones 306 are electricallyconnected to the flexible mother circuit 318, which provides electricalcommunication between the microphones 306 and a digital signal processor(DSP) 312. Among other components, the DSP 312 incorporates or iscoupled to audio signal processing circuitry configured to implementvarious functions described herein. A sensor package 314 can be coupledto the DSP 312 via the flexible mother circuit 318. The sensor package314 can include one or more different specific types of sensors such asthose described in greater detail below. One or more user switches 310(e.g., on/off, volume, mic directional settings) are electricallycoupled to the DSP 312 via the flexible mother circuit 318.

An audio output device 316 is electrically connected to the DSP 312 viathe flexible mother circuit 318. In some embodiments, the audio outputdevice 316 comprises a speaker (coupled to an amplifier). In otherembodiments, the audio output device 316 comprises an amplifier coupledto an external receiver 320 adapted for positioning within an ear of awearer. The external receiver 320 can include an electroacoustictransducer, speaker, or loud speaker. The hearing assistance device 200may incorporate a communication device 308 coupled to the flexiblemother circuit 318 and to an antenna 302 directly or indirectly via theflexible mother circuit 318. The communication device 308 can be aBLUETOOTH® transceiver, such as a BLE (BLUETOOTH® low energy)transceiver or other transceiver (e.g., an IEEE 802.11 compliantdevice). The communication device 308 can be configured to communicatewith one or more external devices, such as those discussed previously,in accordance with various embodiments. In various embodiments, thecommunication device 308 can be configured to communicate with anexternal visual display device such as a smart phone, a video displayscreen, a tablet, a computer, a hologram, a virtual or augmented realitydevice, or the like. In various embodiments, the communication device308 can be configured to communicate with one or more of the controlsystem or navigation system of a vehicle. In various embodiments, thecommunication device 308 can be configured to communicate with one ormore communication devices associated with a third party. In variousembodiments, the hearing assistance device 200 can also include acontrol circuit 322 and a memory storage device 324. The control circuit322 can be in electrical communication with other components of thedevice. The control circuit 322 can execute various operations, such asthose described herein. The control circuit 322 can include variouscomponents including, but not limited to, a microprocessor, amicrocontroller, an FPGA (field-programmable gate array) processingdevice, an ASIC (application specific integrated circuit), or the like.The memory storage device 324 can include both volatile and non-volatilememory. The memory storage device 324 can include ROM, RAM, flashmemory, EEPROM, SSD devices, NAND chips, and the like. The memorystorage device 324 can be used to store data from sensors as describedherein and/or processed data generated using data from sensors asdescribed herein, including, but not limited to, information regardingexercise regimens, performance of the same, visual feedback regardingexercises, and the like.

As mentioned with regard to FIG. 2, the hearing assistance device 200shown in FIG. 2 is a receiver-in-canal type device and thus the receiveris designed to be placed within the ear canal. Referring now to FIG. 4,a schematic view is shown of a hearing assistance device disposed withinthe ear of a subject in accordance with various embodiments herein. Inthis view, the receiver 206 and the earbud 208 are both within the earcanal 112, but do not directly contact the tympanic membrane 114. Thehearing device housing is mostly obscured in this view behind the pinna110, but it can be seen that the cable 204 passes over the top of thepinna 110 and down to the entrance to the ear canal 112.

While FIG. 4 shows a single hearing assistance device, it will beappreciated that subjects can utilize one or more hearing assistancedevices such as two, with one for each ear. In some embodiments, thehearing assistance devices and sensors therein can be disposed onopposing lateral sides of the subject's head. Specifically, the hearingassistance devices and sensors therein can be disposed in a fixedposition relative to the subject's head. In some embodiments, thehearing assistance devices and sensors therein can be disposed withinopposing ear canals of the subject. In some embodiments, the hearingassistance devices and sensors therein can be disposed on or aboutopposing ears of the subject. The hearing assistance devices and sensorstherein can be spaced apart from one another by a distance of at least3, 4, 5, 6, 8, 10, 12, 14, or 16 centimeters and less than 40, 30, 28,26, 24, 22, 20 or 18 centimeters, or by a distance falling within arange between any of the foregoing.

Systems herein, and in particular hearing assistance devices herein, caninclude sensors (such as part of a sensor package 314) to detectmovements of the subject wearing the hearing assistance device.Movements can be detected in any axis and can also specifically includerotation around any axis. Exemplary sensors are described in greaterdetail below. Referring now to FIG. 5, a schematic side view is shown ofa subject 500 wearing a hearing assistance device 200 in accordance withvarious embodiments herein. For example, movements detected can includeforward and/or backward movements 506 of the subject's head 502, upwardand/or downward movements 508 of the subject's head 502, and rotationalmovements 504 of the subject's head 502 in a vertical plane. While FIG.5 shows a single hearing assistance device, it will be appreciated thatin various embodiments, subjects can wear two hearing assistancedevices. Referring now to FIG. 6, a schematic top view is shown of asubject 500 wearing hearing assistance devices 200, 600 in accordancewith various embodiments herein. Movements detected, amongst others, canalso include side-to-side movements 604 of the subject's head 502,rotational movements 602 of the subject's head 502 in the horizontalplane, tilts to either side, and tilting of the subject's head 502 inthe vertical plane. As described above, embodiments of systems herein,such as hearing assistance devices, can track the motion or movement ofa subject using a motion sensor. The head position of the subject canalso be tracked. In various embodiments, the head position of thesubject can be tracked simultaneously with tracking of the motion ormovement of the subject. In various other embodiments, the position ofother parts of the subject's body can also be tracked using operativelyconnected sensors distributed across the body (e.g., a body network ofdevices).

In some embodiments, the system can estimate visual system input basedon the tracked head position. The system can estimate consistencybetween or otherwise compare the vestibular system inputs and the visualsystem inputs. In various embodiments, the system can also initiateresponsive measures if the estimated consistency crosses a thresholdvalue.

Referring now to FIG. 7, a schematic side view is shown of a subject 500riding in a car 702 in accordance with various embodiments herein. Thecar 702 can serve as a specific example of a vehicle. However, as usedherein, the term “vehicle” shall include any machine that transportspeople including, but not limited to, wagons, bicycles, motor vehicles(motorcycles, cars, trucks, buses), railed vehicles (trains, trams),mobility assistance devices (e.g., wheelchairs, motorized wheelchairs,exoskeletons), self-balancing personal transporters, watercraft (ships,boats), amphibious vehicles (screw-propelled vehicle, hovercraft),aircraft (airplanes, helicopters, drones), agricultural and industrialequipment, and spacecraft.

In use, the car 702 can move predominantly in a particular direction ofmotion 704 at varying speeds. A subject 500 riding in the car 702 couldexperience visual system input that is largely consistent to vestibularsystem input, if they were looking straight ahead, such as if they weredriving the car 702. However, particularly as a passenger, the subject500 may be looking downward at a book or magazine, an external displaydevice 706 such as a smartphone, an electronic game, or the like. In sodoing, the subject's line of gaze 708 may deviate from the direction ofmotion 704. The subject's line of gaze 708 can be deemed to be a markerand/or proxy of visual system input in accordance with variousembodiments herein. The direction of motion 704 can be deemed to be amarker and/or proxy of vestibular system input in accordance withvarious embodiments herein.

The direction of motion 704 can be determined by evaluating data fromone or more sensors that are part of a sensor package 314 herein. By wayof example, the direction of motion 704 can be determined by evaluatingdata from one or more motion sensors. The direction of motion 704 canalso be determined by operatively connected sensors in the vehicle(which can be part of safety/guidance systems of the vehicle) in or onwhich the subject is riding (e.g., a signal from the vehicle reportingsensor data indicative of vehicle movement). The direction of motion 704can also be determined by operatively connected sensors in anotherdevice, such as the smartphone of the user. In addition, futuredirection of motion can be predicted based upon maps, global positioningsystems, navigation systems, and by the safety/guidance systems of thevehicle. Future direction of motion may also be predicted by monitoringthe gaze (visual hotspots) of the wearer, of by head movements of thewearer (e.g., head tilting prior to a turn).

In various embodiments, he subject's line of gaze 708 can be determinedby calculating the rotational position of the subject's head 502 withrespect to gravity and/or with respect to the determined direction ofmotion 704. In various embodiments, the subject's line of gaze 708 canalso be determined by tracking eye movement from an ear-level device.Exemplary techniques for tracking eye movement from an ear-level deviceare described in U.S. Pat. No. 9,167,356, issued Oct. 20, 2015, entitledELECTROOCULOGRAM AS A CONTROL IN A HEARING ASSISTANCE DEVICE, thecontent of which is herein incorporated by reference. In variousembodiments, the subject's line of gaze 708 can also be determined byusing other operatively connected eye tracking devices, such as camerasfacing an individual's eyes. The individual could be the subject oranother individual such as one who is driving a vehicle (e.g., a subjectas a passenger in the back of car can benefit from the directiondetermination/prediction that is enabled by tracking the eyes of thedriver of a vehicle.

The subject's line of gaze 708 may deviate from the direction of motion704 by angle θ¹. In accordance with various embodiments herein, thesystem can detect angle θ¹. In accordance with various embodimentsherein, angle θ¹ can be used (by itself or in combination with otherdata inputs) to provide an estimation of inconsistency between visualsystem inputs and vestibular system inputs.

In some embodiments, the specific amount of degrees by which thesubject's direction of gaze differs from the direction of movement (forexample, angle θ¹) can be captured and used in an evaluation ofconsistency between vestibular system and visual system inputs. In someembodiments, evaluation of whether a specific threshold value has beencrossed for the difference in direction between the subject's directionof gaze and the direction of movement can be used in an evaluation ofconsistency between vestibular system and visual system inputs.Threshold values can be about 1, 2, 3, 5, 7, 10, 15, 20, 25, 30, 35, 40,45, 50, 60 degrees or more, or can be an amount falling within a rangebetween any of the foregoing.

A subject's head tends to move over time. Thus, while angled downward θ¹for a period of time, the subject may periodically reposition theirhead—(e.g. positioning at an upward angle then re-positioning at adownward angle). However, in some scenarios, transitory head movementsmay not impact motion sickness significantly. As such, in someembodiments, the amount of time and/or percentage of time that thesubject's direction of gaze is at a particular angle with respect to thedirection of movement can be tracked and evaluated in accordance withembodiments herein.

In some embodiments, a threshold may be set for a period of time duringwhich the subject's direction of gaze deviates from the direction ofmovement before the deviation is considered as part of the evaluation ofconsistency between vestibular system and visual system inputs. In someembodiments, the threshold can be greater than or equal to 1, 3, 5, 7,9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43,45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71, 73, 75, 77, 79,81, 83, 85, 87, 89, 91, 93, 95, 97, 99, 100, 110, 130, 180, 240 or 360seconds, or can be an amount falling within a range between any of theforegoing.

In some embodiments, a histogram can be prepared including informationon the total amount of time and/or percentage of time that the subject'sdirection of gaze deviates from the direction of movement by specificamounts or amount falling within specific ranges.

It will be appreciated that a subject's line of gaze can be differentthan the direction of motion in other than a vertical dimension. Forexample, an individual riding in a vehicle may be looking to the sidewith respect to the direction of motion. Referring now to FIG. 8, aschematic top view is shown of a subject 500 riding in a car 702 inaccordance with various embodiments herein.

The subject's line of gaze 708 may deviate from the direction of motion704 in a horizontal plane by angle θ². In accordance with variousembodiments herein, the system can detect angle θ². In accordance withvarious embodiments herein, angle θ² can be used as one element of anestimation of inconsistency between visual system inputs and vestibularsystem inputs.

In some embodiments, the specific amount of degrees by which thesubject's direction of gaze differs from the direction of movement inthe horizontal plane can be captured and used in an evaluation ofconsistency between vestibular system and visual system inputs. In someembodiments, whether or not the subject's direction of gaze differs fromthe direction of movement in the horizontal plane by at least a specificthreshold value can be used in an evaluation of consistency betweenvestibular system and visual system inputs. Threshold values can beabout 1, 2, 3, 5, 7, 10, 15, 20, 25, 30, 35, 40, 45, 50, 60, 70, 80 or90 degrees or more, or can be an amount falling within a range betweenany of the foregoing. In some embodiments, the subject 500 could beseated backwards with respect to the direction of movement (such as whenseated on a train). As such, in some embodiments, the threshold can beabout 90, 100, 110, 120, 130, 140, 150, 160, 170 or 180 degrees, or anamount falling within a range between any of the foregoing.

A subject's head may tend to move over time. Thus, while angled in thevertical plan (to the side) θ² for a period of time, the subject mayperiodically move their head from side to side. As such, in someembodiments, the amount of time and/or percentage of time that thesubject's direction of gaze is at a particular angle in the horizontalplane with respect to the direction of movement can be tracked andevaluated in accordance with embodiments herein.

In some embodiments, a threshold may be set for a period of time duringwhich the subject's direction of gaze deviates from the direction ofmovement in the horizontal plane before the deviation is considered aspart of the evaluation of consistency between vestibular system andvisual system inputs. In some embodiments, the threshold can be greaterthan or equal to 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29,31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65,67, 69, 71, 73, 75, 77, 79, 81, 83, 85, 87, 89, 91, 93, 95, 97, 99, 100,110, 130, 180, 240 or 360 seconds, or can be an amount falling within arange between any of the foregoing.

When a subject is riding in a vehicle and their head is angled downward,the subject's focal distance may be much different and much closer thanif their head is up (in a neutral position) and they are looking in thedirection of movement. For example, the subject's focal depth may be adistance of inches to feet (such as 8 inches to 3 feet) when focused onan object in the car versus a much greater distance (such as 10 yards to∞) if their eyes are focused on objects outside the car in the directionthe vehicle is traveling such as the road ahead and/or the horizon. Invarious embodiments herein, the subject's focal depth can be estimatedbased on factors including the current rotational position of their headin a vertical plane (e.g., head angled downward, level, or angledupward, and/or degree of same), current rotational position in ahorizontal plane (e.g., looking to the side such as out a window orstraight ahead), and, in some cases, information regarding the currentsurroundings of the subject (with can be identified/classified asdescribed further below). For example, if the subject's head is angleddown, but in the horizontal plane is straight ahead and the currentsurroundings are classified as a car, a bus, or a train, then it can bepresumed that the current focal depth is on the order of 8 inches to 3feet.

Motion sickness may be commonly experienced while a subject is riding invarious types of vehicles. Some vehicles, in particular those fortraversing water, can result in types of movement that is particularlylikely to trigger motion sickness symptoms in some individuals. Systemsin accordance with embodiments herein can be used to prevent and/ormitigate motion sickness in a subject riding in or on such vehicles.

Referring now to FIG. 9, a schematic side view is shown of a subject 500riding in a boat 902 in accordance with various embodiments herein. Theboat 902 can move in a main direction of movement 906. However, the boat902 can also move vertically 904. The boat 902 can also roll 908 and canpitch 910. As such, a subject 500 riding in or on the boat 902 can besubjected to movement vertically 904 as well as roll 908 and pitch 910.Simultaneously, the hearing assistance device(s) 200 and sensors thereincan be used to track the position of the subject's head as well as thedirection of motion 704, the subject's line of gaze 708, the degree andfrequency of roll, the degree and frequency of pitch, the degree andfrequency of vertical movement (such as vertical oscillating movementassociated with a vehicle traversing water), and the like.

In various embodiments, external visual display devices can be used aspart of systems and methods herein. The external visual display devicecan include a display screen and a camera. In some embodiments, theexternal visual display device can be a smartphone, a video monitor, avirtual reality display device, an augmented reality display device, orthe like.

In some embodiments, the display screen can be a video touch screen. Thedisplay screen can display various pieces of information to the subjectincluding, but not limited to, an indication that motion sicknessinducing conditions have been detected, a query to the subject regardinghow they feel, instructions for actions the subject should take toreduce the chances that they will experience motion sickness,instructions for action the subject should take to mitigate motionsickness if onset of the same has already occurred, a query to thesubject regarding the effectiveness of a responsive measure, or thelike.

The display screen can also provide the user with stimulation thatallows the user to visualize motion in accordance with the user'svestibular system. For example, when the user is inside a movingvehicle, the inside of the vehicle may appear visually stationary to theuser, which may be in disagreement with the vestibular input that theuser may also be receiving. In at least one example, the system maydisplay video images or an animation on a display screen of a devicethat simulates motion consistent with the motion of the user. Suchimages or an animation can also be displayed using a virtual oraugmented reality display. The displayed video imagery may be furtherresponsive to characteristics of the user's motion, such as changes inspeed, changes in one or more of an altitude or distance relative to thesurfaces approximate to the vehicle, changes in color, texture orlighting relative to the surfaces approximate to the vehicle, and thelike. The displayed video imagery may be further responsive to predictedfuture changes of direction, speed, altitude, distance, surfacecharacteristics, etc. of the vehicle. The displayed video imagery may beresponsive to measured or reported severity of motion sickness symptoms.In addition, the display device may contain a motion sensor anddynamically adapt the perspective of the image being displayed, e.g.,the motion simulation video or animation relative to the position of thedisplay, itself. In at least one example, the display screen may show avideo or animation that simulates motion in accordance with the motionof the user such that other displays (e.g., text, apps, games, images,videos, webpages, and the like) are overlaid/superimposed with a degreeof opacity over top of the pictured video or animation simulation.Exemplary opacity values can be about 5%, 15%, 25%, 35%, 45%, 55%, 65%,75%, 85%, or 95%, or can be an amount falling within a range between anyof the foregoing. In at least one embodiment, the opacity of the overlaycan be adjusted by the user. In at least one other embodiment, theopacity of the overlay may be dynamically adapted by one or morecomponents of the system and/or control circuitry thereof. In someembodiments, a motion simulation video or animation may be presented tothe user as a virtual or augmented reality.

A video processing circuit (which can be a part of a device with adisplay screen herein) can generate a 2D or 3D image based oninformation including one or more of geometry, viewpoint, texture,lighting and shading information, overlay graphics, and the like. Insome embodiments, the video processing circuit can include a GPU(graphical processing unit). The term “graphics pipeline” or “viewrendering pipeline” can be used to refer to the sequence of steps usedto create 2D graphical images or 2D raster representations of a 3Dscene. The video processing circuit and/or GPU can execute one or moresteps of the graphics pipeline. The video processing circuit and/or GPUcan also include one or more physical components used in the graphicspipeline. The graphics pipeline can include one or more stages ofcreating a scene out of geometric primitives, modelling andtransformation, camera transformation, lighting, projectiontransformation, clipping, scan conversion or rasterization, andtexturing and fragment shading. In various embodiments, other operationscan also be performed. In various embodiments, the graphics pipeline canuse OpenGL, DirectX, or other protocols.

The camera of the external visual display device can be positioned toface away from the display screen and back toward the subject. Thecamera can be used to capture an image or images of the subject's faceand, in some cases, the subject's eyes. In some embodiments, the cameracan be used to capture image(s) including the positioning of a subject'sface, pupil, iris, and/or sclera. Such information can be used tocalculate the direction of the subject's face and/or gaze. In someembodiments, such information can also be used to calculate angle, speedand direction of nystagmus. In some embodiments, information aboutnystagmus observed from the user's eyes may be used to further calculateor estimate the user's vestibular system inputs to the brain of theuser. Aspects of nystagmus detection and characterization are describedin commonly-owned U.S. Publ. Pat. Appl. No. 2018/0228404, the content ofwhich is herein incorporated by reference. In some embodiments, suchinformation can specifically be used to calculate the direction of thesubject's face and/or gaze with respect to the camera. Aspects regardingsuch calculations are described in U.S. Publ. Appl. Nos. 2012/0219180and 2014/0002586; the content of which is herein incorporated byreference.

Referring now to FIG. 10, a schematic view is shown of an externaldisplay device 706 and elements of a display screen 1004 thereof inaccordance with various embodiments herein. The external display device706 can include a camera 1006, a speaker 1008 and the like. Many visualdisplay options are contemplated herein. In various embodiments, theexternal display device 706 can display an indicator 1010 or warningthat conditions associated with motion sickness have been detected. Invarious embodiments, the external display device 706 can display a query1012 for the subject. For example, the query 1012 could relate to howthe subject is currently feeling. In some embodiments, the system can beconfigured to display the query 1012 regarding how the subject isfeeling after detecting conditions associated with motion sickness. Insome embodiments, the external display device 706 can display virtualbuttons 1014, 1016 (or input icons) to accept user input from thesubject through the display screen 1004. It will be appreciated thatqueries to the subject can also be aural. For example, a hearingassistance device or an external device can present an auditory/verbalquery. Also, feedback from the subject can be provided verbally orthrough gestures (e.g., head nodding, etc.) which can be detected by amicrophone and/or motion sensors of the hearing assistance device or anexternal device.

In some embodiments, user input from the subject can regard at least oneof a start, an end, a presence of, or an intensity of a motion sicknessepisode. In various embodiments herein, the estimation model forvestibular system input and visual system input can be altered based onthe subject input. In various embodiments, the threshold value forconsistency between the vestibular system input and the visual systeminput can be altered based on the user input from the subject.

In some embodiments, the system can detect emesis of the subject. Emesiscan be detected based on characteristic acoustic patterns and/orcharacteristic postural and diaphragm/abdominal contraction patterns. Invarious embodiments, emesis can be detected by analyzing signals from atleast one of an IMU and a microphone.

Methods

Various methods are included herein. Methods herein can include methodsof preventing or mitigating motion sickness in a subject. Methods hereincan include the execution of operations described elsewhere herein.

In a specific embodiment, a method of preventing or mitigating motionsickness in a subject is included herein. The method can includetracking motion of the subject using a first motion sensor. Exemplarymotion sensors are described in greater detail below.

The method can further include estimating vestibular system input basedon tracked motion of the subject. In some embodiments, an estimate ofvestibular system input can be formed through applying a functionrelating tracked motion of the subject and vestibular system input.Aspects of the function can include one or more of the speed, velocity,consistency of speed and velocity, acceleration, consistency ofacceleration, movement in directions other than the predominantdirection of movement (e.g., such as bumps or other vertical or lateralmovement experienced while riding in or on a vehicle), and the like. Insome embodiments, the function relating tracked motion of the subject tovestibular system input can be derived empirically based on a populationof subjects. In some embodiments, the function relating tracked motionof the subject to vestibular system input can be derived empiricallybased on a population of subjects with vestibular conditions similar tothe subject using the device including factors relevant to motionsickness such as age, gender, health status, inner ear function, and thelike. In some embodiments, the function relating tracked motion of thesubject to vestibular system input can be derived through a calibrationprocedure executed with the subject using the device. By way of example,calibration can include prompting the subject to look at the horizon inthe same direction as motion and/or looking above or below the horizonand/or at an angle with respect to the direction of motion. In variousembodiments, the function can filter out movement having a frequencyabove a threshold value (digitally or using a low pass filter or thelike).

In various embodiments, the method can also include tracking headposition of the subject using the first motion sensor. Exemplary sensorsused for tracking head position are described in greater detail below.

In various embodiments, the method can also include estimating visualsystem input based on tracked head position of the subject. In variousembodiments, the method can also include estimating consistency betweenthe vestibular system input and the visual system input. In variousembodiments, the method can also include initiating responsive measuresif the estimated consistency crosses a threshold value.

Sensors

Systems herein can include one or more sensor packages to provide datain order to determine aspects including, but not limited to, trackingmovement of a subject and tracking head position of the subject. Thesensor package can comprise one or a multiplicity of sensors. In someembodiments, the sensor packages can include one or more motion sensorsamongst other types of sensors. Motion sensors herein can includeinertial measurement units (IMU), accelerometers, gyroscopes,barometers, altimeters, and the like. Motions sensors can be used totrack movement of a subject in accordance with various embodimentsherein.

In some embodiments, the motion sensors can be disposed in a fixedposition with respect to the head of a subject, such as worn on or nearthe head or ears. In some embodiments, the motion sensors can bedisposed associated with another part of the body such as on a wrist,arm, or leg of the subject.

Sensor packages herein can also include one or more of a magnetometer,microphone, acoustic sensor, electrocardiogram (ECG),electroencephalography (EEG), eye movement sensor (e.g.,electrooculogram (EOG) sensor), myographic potential electrode (EMG),heart rate monitor (photoplethysmogram, electrocardiogram, etc.), pulseoximeter, magnetic sensor, a telecoil, a wireless radio antenna, abarometer, thermometer, and the like.

In some embodiments, the sensor package can be part of a hearingassistance device. However, in some embodiments, the sensor packages caninclude one or more additional sensors that are external to a hearingassistance device. The one or more additional sensors can comprise oneor more of an IMU, accelerometer, gyroscope, barometer, magnetometer, anacoustic sensor, eye motion tracker, EEG or myographic potentialelectrode (e.g., EMG), heart rate monitor (photoplethysmogram,electrocardiogram, etc.), and pulse oximeter. For example, the one ormore additional sensors can include a wrist-worn or ankle-worn sensorpackage, a sensor package supported by a chest strap, or a sensorpackage in the form of a pendant that hangs from the neck. In someexamples, the one or more additional sensors may be packaged as part ofan accessory device to the hearing assistance device, such as a wirelessaudio streaming device, telephone or landline streamer, remote control,Direct Audio Input (DAI) gateway, remote microphone, telecoil receiver,captioning device, wearable or implantable health monitor, andcombinations thereof.

The sensor package of a hearing assistance device can be configured tosense motion of the wearer. Data produced by the sensor(s) of the sensorpackage can be operated on by a processor of the device or system.

According to various embodiments, the sensor package can include one ormore of an IMU, and accelerometer (3, 6, or 9 axis), a gyroscope, abarometer, an altimeter, a magnetometer, an eye movement sensor, apressure sensor, an acoustic sensor, a heart rate sensor, an electricalsignal sensor (such as an EEG, EMG or ECG sensor), a temperature sensor,a blood pressure sensor, an oxygen saturation sensor, an optical sensor,a thermometer, a global positioning system (GPS) sensor, and the like.In at least one embodiment, the sensor package may be integrated intothe vehicle in/on which the user is riding and operatively connected tothe hearing assistance device.

As used herein the term “inertial measurement unit” or “IMU” shall referto an electronic device that can generate signals related to a body'sspecific force and/or angular rate. IMUs herein can include one or moreof an accelerometer (3, 6, or 9 axis) to detect linear acceleration anda gyroscope to detect rotational rate. In some embodiments, an IMU canalso include a magnetometer to detect a magnetic field.

The eye movement sensor may be, for example, an electrooculographic(EOG) sensor, such as an EOG sensor disclosed in commonly owned U.S.Pat. No. 9,167,356, which is incorporated herein by reference. Thepressure sensor can be, for example, a MEMS-based pressure sensor, apiezo-resistive pressure sensor, a flexion sensor, a strain sensor, adiaphragm-type sensor and the like.

The temperature sensor can be, for example, a thermistor (thermallysensitive resistor), a resistance temperature detector, a thermocouple,a semiconductor-based sensor, an infrared sensor, or the like.

The blood pressure sensor can be, for example, a pressure sensor. Theheart rate sensor can be, for example, an electrical signal sensor, anacoustic sensor, a pressure sensor, an infrared sensor, an opticalsensor, or the like.

The oxygen saturation sensor can be, for example, an optical sensor, aninfrared sensor, or the like.

The electrical signal sensor can include two or more electrodes and caninclude circuitry to sense and record electrical signals includingsensed electrical potentials and the magnitude thereof (according toOhm's law where V=IR) as well as measure impedance from an appliedelectrical potential.

The sensor package can include one or more sensors that are external tothe hearing assistance device. In addition to the external sensorsdiscussed hereinabove, the sensor package can comprise a network of bodysensors (such as those listed above) that sense movement of amultiplicity of body parts (e.g., arms, legs, torso).

Surroundings Classification

In various embodiments of systems and methods herein, the system canclassify the current surroundings of the subject. Classifying thecurrent surroundings of the subject can be useful for purposes ofselecting appropriate guidance to provide to the subject to prevent ortreat symptoms of motion sickness. Classifying the current surroundingsof the subject can also be useful for purposes of appropriatelygenerating virtual or augmented reality visual and/or aural stimuli toprevent or treat symptoms of motion sickness. For example, when the useris in the car, a visual display can play a video animation depicting theroad beneath the car, in motion relative to the movement of thevehicle/user. For example, an image of the road showing the paintedlines passing beneath could be provided.

In various embodiments, the system can classify the current surroundingsof the subject as one of vehicular, stationary, and non-vehicular,non-stationary. Vehicular surroundings can include car, truck, passengervehicle, bus, train, plane, boat or the like. In many cases of vehicularsurroundings causing motion sickness, the vestibular system may indicatemovement, but the visual system may indicate no movement resulting in aninconsistency between the vestibular system input and the visual systeminput.

Examples of stationary surroundings can include a theater, a virtualreality environment, or the like. In many cases of stationarysurroundings causing motion sickness, the vestibular system may indicateno movement, but the visual system may indicate movement is takingplace. Non-vehicular, non-stationary surroundings can include, forexample, a carnival ride, playground equipment, elevators/escalators,building sway, sports event participation, and the like. Examples ofvehicular, stationary, and non-vehicular, non-stationary are shown belowin Table 1.

TABLE 1 Non-Vehicular, Non- Vehicular Stationary Stationary PassengerVehicle VR Usage Carnival Rides (Car/Truck/Etc.) Theater Playground BusPresence in Front Equipment Train of Screen Earthquake or Plane TremorsBoat Elevators/Escalators Moving walkways Building Sway Sporting EventParticipation

For the surroundings to be classified as vehicular, data inputs(including, but not limited to, sensor data) can be evaluated todetermine whether they are consistent with the subject being in avehicle. If the surroundings are non-consistent with a vehicularscenario, then the surroundings can be classified as either stationaryor non-stationary. Exemplary data inputs consistent with vehicular,stationary, and non-vehicular, non-stationary are shown below in Table2.

TABLE 2 Non-Vehicular, Non- Vehicular Stationary Stationary NetDirectional Lack of Significant Rotational Motion Motion Exceeding NetDirectional without Significant a Threshold Motion Net DirectionalMagnitude Lack of Significant Movement Detection of RotationalOscillating Motion Wireless Movement without Significant CommunicationConsistent Net Directional Signals Consistent Direction of Eye Movementwith a Vehicle Gaze Detection of Detection of Detection of VibrationsVibrations/Sounds Magnetic Fields Consistent with a Consistent with aConsistent with a Non-Vehicular Vehicle Stationary Event Event Detectionof Detection of acoustics consistent Magnetic Fields with a vehicleConsistent with a Detection of Non-Vehicle Event Magnetic FieldsConsistent with a Vehicle

With regarding to vehicular surroundings, net directional motion canexceed a threshold magnitude of speed. For example, detecting netdirectional motion at a speed of greater than 10, 20, 30, 50, 75 or 100m/s can be indicative of traveling in a vehicle. In some cases, wirelesscommunication signals can be detected and can be used to classify thesurroundings as vehicular. For example, in some cases the system canpickup BLUETOOTH® or WIFI® protocol communications and determine whetherthe same are consistent with a vehicle or vehicle navigation system.

In some cases, the system can further subclassify the surroundings. Byway of example, if the surroundings are identified as being vehicular,the system can further identify a particular type or class of vehicle.This can be useful for purposes of selecting appropriate guidance toprovide to the subject to prevent or treat symptoms of motion sickness.

In some cases, the system can detect vibrations and/or sounds that areconsistent with vehicles, such as vibrations associated with an engineor movement of tires along pavement or a train along train tracks. Byway of example, the frequencies of road noise associated with passengervehicles generally display a prominent peak between 700 and 1300 Hz, andmore particularly around 1000 Hz. By way of example, a train in motionapplies moving steady loads to the railway track as well as dynamicexcitation; this causes track deflections, vibration and noise. Atgenerally lower frequencies than with passenger vehicles such as carsand trucks, the spectrum of measured track vibration has a distinctpattern, with spectral peaks occurring at multiples of the vehiclepassing frequency. This pattern can be identified in order to classifythe type of vehicle as a train.

In some embodiments, detection of sustained oscillating motion at afrequency of less than 20 Hz, 10 Hz, 5 Hz, 2 Hz, 1 Hz, or 0.5 Hz resultsin a classification of current surroundings as vehicular and asubclassification as a boat. In some embodiments, detection of sustainedsound with a prominent peak at a frequency of less than 700, 500, 300,or 200 Hz results in a classification of current surroundings asvehicular and a subclassification as a train. In some embodiments,detection of sustained motion at a speed of greater than 120, 135, 150,180, or 200 MPH results in a classification of current surroundings asvehicular and a subclassification as a plane. In some embodiments,detection of a change in altitude exceeding a threshold value over lessthan a threshold amount of time results in a classification of currentsurroundings as vehicular and a subclassification as a plane. In someembodiments, detection of altitude changes exceeding 300, 500, 750,1000, or 1200 ft/min results in a classification of current surroundingsas vehicular and a subclassification as a plane. In some embodiments,detection of altitude changes exceeding 300, 500, 750, 1000, or 1200ft/min accompanied by a total altitude change of greater than 2000,2500, 3000, 4000, 5000, 7500, or 10000 feet results in a classificationof current surroundings as vehicular and a subclassification as a plane.

In some embodiments, the system can detect magnetic field strengths,frequencies, or patterns that are consistent with particularenvironments, including vehicles. By way of example, the electricalenergy of a spark plug used in an internal combustion engine can createa detectable magnetic field. A detectable magnetic field of anautomobile generally pulses at rate in accordance with the revolutionsof the vehicle's engine. Under typical operating circumstances, apassenger vehicle has a periodic, wideband magnetic field pulsefrequency of 50 to 350 times each second. The frequency spectrum of themagnetic field energy of a passenger vehicle typically exhibits severalharmonics with a fundament frequency. Each vehicle has its ownidentifiable characteristic engine behavior which is influenced by howthe vehicle's manufacturer has designed the operation of the engine,electric motor, drivetrain, alternator, ventilation system, onboardelectronics, and the like.

Exemplary data inputs consistent with specific types/classes of vehicleare shown below in Table 3.

TABLE 3 Passenger Vehicle Bus Train Plane Boat Sound Sound Sound Netdirectional Low frequency frequencies frequencies frequencies motiongreater oscillating with prominent with prominent with prominent than150 mph; motion (pitch, peak between peak between peak below Altituderoll, vertical 700 and 1300 700 and 1300 500 Hz. changes oscillation);Hz. Hz. exceeding 500 Net directional ft/min motion less than 20 m/s.Responsive and/or Preventative Measures

The system can be configured to initiate responsive measures ifconditions for motion sickness are detected. The system can beconfigured to initiate responsive measures if the estimated consistencybetween the vestibular system input and the visual system input crossesa threshold value.

In various embodiments, the responsive measures can include promptingthe subject to move their head sufficiently to increase consistencybetween the vestibular system input and the visual system input. Invarious embodiments, the responsive measure can include lifting thesubjects head to a horizontal position if the tracked head position isindicated to be below or above horizontal. In various embodiments, theresponsive measure can include administering aural stimulation to thesubject. In various embodiments, the responsive measure can includeprompting the subject to open a vent and/or adjust a fan speed toincrease air flow. In various embodiments, the responsive measure caninclude prompting the subject to maintain their gaze at a fixed point.In various embodiments, the responsive measure can include prompting thesubject to maintain their gaze at a fixed point on the horizon no lessthan 30 degrees different than the direction of a vector representingtheir current motion. In various embodiments, the responsive measure caninclude prompting, pacing, or otherwise assisting the subject to breathaccording to a predetermined cadence. In some embodiments, thepredetermined cadence can be less than or equal to 18, 16, 14, 12, 10,8, 6, or 4 breaths per minute, or can be an amount falling within arange between any of the foregoing.

In various embodiments, the responsive measures can include providingauditory stimulation. In various embodiments, auditory stimulation caninclude a perceptually stationary auditory stimulus. In variousembodiments, the responsive measures can include prompting the subjectto focus their gaze on the horizon. In various embodiments, can usetracked head position of the subject to assess whether the subjectfocused their gaze on the horizon.

In accordance with various embodiments herein, hearing assistancedevice(s) that can be configured to guide the wearer of a hearingassistance device through a prescribed series of body movements oractions in accordance with a predetermined corrective or therapeuticmaneuver, physical therapy or exercise routine. A maneuver, physicaltherapy or exercise routine involves a prescribed series of bodymovements or actions that can be implemented by the wearer of a hearingassistance device in an attempt to correct or treat a physiologicdisorder or execute a physical fitness routine. Exercises (or routinesor maneuvers herein) can include, but are not limited to, habituationexercises, gaze stabilization exercises, and balance training exercises.Exercises can include a series of actions including one or more ofturning their head in a specified direction by a specified amount,moving their head in a specific direction by a specified amount,focusing their gaze on a stationary or moving point, assuming differentpostures, etc.

Guidance and/or feedback herein can include auditory guidance, visualguidance, or auditory and visual guidance. Audio guidance can includeany one or a combination of different sounds, such as tones, noisebursts, human speech, animal/natural sounds, synthesized sounds, andmusic, among other sounds.

In some embodiments, a virtual audio interface can be used to provideauditory stimulation to a subject. The virtual audio interface can beconfigured to synthesize three-dimensional (3-D) audio that guides thewearer in performing specific physical movements of a predeterminedcorrective or therapeutic maneuver, physical therapy or exerciseroutine. In specific, the virtual audio interface can be configured tosynthesize three-dimensional (3-D) audio that guides the wearer toperform physical movements to prevent and/or alleviate symptoms ofmotion sickness. For example, a synthesized 3-D virtual audio target canbe generated at the specified location relative to the wearer's currenthead position. In response, the wearer moves his or her head in thespecified direction indicated by the audio target.

According to some embodiments, the virtual audio interface can generateaudio cues comprising spatialized 3-D virtual sound emanating fromvirtual spatial locations that serve as targets for guiding wearermovement. In some embodiments, the spatialized 3-D virtual sound can begenerated so as to be perceived by the wearer as emanating from thehorizon. The horizon is perpendicular to the direction of gravity. Thedirection of gravity can be measured by accelerometer as a constant biasin that direction. Therefore, signals from the accelerometer can be usedto determine the location of the horizon.

The wearer can execute a series of body movements in a direction and/orextent indicated by a sequence of virtual sound targets. The soundgenerated at the virtual spatial locations can be any broadband sound,such as complex tones, noise bursts, human speech, music, etc. or acombination of these and other types of sound. In various embodiments,the virtual audio interface is configured to generate binaural ormonaural sounds, alone or in combination with spatialized 3-D virtualsounds. The binaural and monaural sounds can be any of those listedabove including single-frequency tones.

In other embodiments, the virtual audio interface is configured togenerate human speech that guides the wearer in performing specificphysical movements of a predetermined corrective or therapeuticmaneuver, such as to prevent or alleviate motion sickness. The speechcan be synthesized speech or a pre-recording of real speech. Inembodiments that employ a single hearing assistance device (for oneear), for example, the virtual audio interface generates monaural soundin the form of speech, which can be accompanied by other sounds, such assingle or multi-frequency tones, noise bursts or music. In embodimentsthat employ two hearing assistance devices (one device for each ear),the virtual audio interface can generate monaural or binaural sound inthe form of speech, which can be accompanied by other sounds, such assingle or multi-frequency tones, noise bursts or music. The virtualaudio interface can display (play back) spoken instructions to guide thewearer though specific physical movements of a predetermined correctiveor therapeutic maneuver, physical therapy or exercise routine. Furtheraspects of virtual audio interfaces are described in commonly owned U.S.patent application Ser. No. 15/589,298, titled “Hearing AssistanceDevice Incorporating Virtual Audio Interface for Therapy Guidance”, thecontent of which is herein incorporated by reference in its entirety.Exemplary speech can include instructions for the wearer to “focus eyeson the horizon”, “pickup your head”, “look straight ahead”, and thelike.

In some examples, systems herein can apply a machine learning algorithmusing reinforcement machine learning techniques to select the bestpossible responsive measure for one or more of the user or a contextregarding the user (e.g., vehicle, non-vehicle, stationary, etc.).Reinforcement learning models can consider one or more of a data input,data classification, data output, responsive measure, guidance,feedback, response to queries, and the like. In one or more examples,the reinforcement machine learning model(s) can further consider thestatistical data associated with historical data (e.g., data stored in adata storage system) to optimize the responsive measure. In one or moreembodiments herein, determining a responsive measure includes applyinginput data to a statistical model or module. Such a statistical model ormodule can be developed from evaluating the effectiveness of responsivemeasures of the hearing assistance device(s) over time.

Further Embodiments

In an embodiment, a method of preventing or mitigating motion sicknessin a subject is included, the method including tracking motion of thesubject using a first motion sensor, estimating a vestibular systeminput based on tracked motion of the subject, tracking head position ofthe subject using the first motion sensor, estimating a visual systeminput based on tracked head position of the subject, estimatingconsistency between the vestibular system input and the visual systeminput, and initiating a responsive measure if the estimated consistencycrosses a threshold value.

In an embodiment, the responsive measure can include prompting thesubject to move their head sufficiently to increase consistency betweenthe vestibular system input and the visual system input.

In an embodiment, the responsive measure can include providing auditorystimulation.

In an embodiment, the auditory stimulation can include a perceptuallystationary auditory stimulus.

In an embodiment, the responsive measure can include prompting thesubject to focus their gaze on the horizon.

In an embodiment, the method can further include using tracked headposition of the subject to assess whether the subject focused their gazeon the horizon.

In an embodiment, the responsive measure can include displaying videoimages on a device with a display screen showing motion consistent withtracked motion.

In an embodiment, the shown motion changes in response to tracked motionchanges.

In an embodiment, estimating visual system input based on tracked headposition of the subject includes estimating a direction of visual focusof the subject.

In an embodiment, a method further can include estimating focal depth ofthe subject.

In an embodiment, a method further can include tracking eye movement ofthe subject.

In an embodiment, estimating vestibular system input based on trackedmotion of the subject includes estimating a motion vector.

In an embodiment, a method further includes estimating vestibular systeminput based on tracked motion of the subject and tracked head positionof the subject.

In an embodiment, a method further can include querying the subjectregarding their status if the estimated consistency crosses a thresholdvalue.

In an embodiment, a method further can include querying the subjectregarding their status if the estimated consistency crosses a thresholdvalue for a threshold amount of time.

In an embodiment, a method further can include sensing ambient soundswith a microphone as part of a hearing assistance device and classifyingcurrent surroundings of the subject based on the ambient sounds.

In an embodiment, a method further can include sensing data with atleast one of a magnetic sensor, a telecoil, a wireless radio antenna,and a motion sensor as part of a hearing assistance device andclassifying current surroundings of the subject based on the ambientsounds.

In an embodiment, a method further can include sensing data with atleast one sensor and further can include classifying the currentsurroundings as one of vehicular, stationary, and non-stationary.

In an embodiment, a method further can include subclassifying vehicularsurroundings as one of a passenger vehicle, a bus, a train, a plane, anda boat.

In an embodiment, a method further can include detecting motion at afrequency of less than 10 Hz.

In an embodiment, a method further can include detecting motion at afrequency of less than 2 Hz.

In an embodiment, a method further can include detecting motion at afrequency of less than 0.5 Hz.

In an embodiment, detection of sustained oscillating motion at afrequency of less than 0.5 Hz results in a classification of currentsurroundings as vehicular and a subclassification as a boat.

In an embodiment, a method further can include detecting ambient sound,wherein detection of sustained sound at a frequency of less than 200 Hzresults in a classification of current surroundings as vehicular and asubclassification as a train.

In an embodiment, detection of sustained motion at a speed of greaterthan 150 MPH results in a classification of current surroundings asvehicular and a subclassification as a plane.

In an embodiment, detection of a change in altitude exceeding athreshold value over less than a threshold amount of time results in aclassification of current surroundings as vehicular and asubclassification as a plane.

In an embodiment, the change in altitude is detected with a barometer.

In an embodiment, a method further can include accepting a subject inputregarding at least one of a start, an end, a presence of, or anintensity of a motion sickness episode.

In an embodiment, a method further can include detecting emesis of thesubject.

In an embodiment, emesis is detected using at least one of an IMU and amicrophone.

In an embodiment, a method further can include altering an estimationmodel for vestibular system input and visual system input based on thesubject input.

In an embodiment, a method further can include altering a thresholdvalue for consistency between the vestibular system input and the visualsystem input based on the subject input.

In an embodiment, a method further can include calibrating the firstmotion sensor including prompting the subject to look at the horizon.

In an embodiment, the first motion sensor is disposed in a fixedposition relative to the subject's head.

In an embodiment, a method further can include tracking motion of thesubject using a second motion sensor and tracking head position of thesubject using the second motion sensor.

In an embodiment, the first motion sensor and the second motion sensorare disposed on opposing lateral sides of the subject's head.

In an embodiment, the first motion sensor and the second motion sensorare disposed on or about opposing ears of the subject.

In an embodiment, the first motion sensor is mounted in a first hearingassistance device and the second motion sensor is mounted in a secondhearing assistance device.

In an embodiment, the first motion sensor can include an IMU.

In an embodiment, a method of preventing or mitigating motion sicknessin a subject is included, the method can include tracking motion of thesubject using a first motion sensor associated with a hearing assistancedevice, tracking head position of the subject using the first motionsensor, sensing ambient sounds with the hearing assistance device,classifying the surroundings based on the tracked motion, tracked headposition, and ambient sound as one of vehicular, stationary, andnon-stationary, initiating a responsive measure based on theclassification of the surroundings.

In an embodiment, a responsive measure can include lifting the subjectshead to a horizontal position if the tracked head position is indicatedto be below or above horizontal.

In an embodiment, a responsive measure can include administering auralstimulation to the subject.

In an embodiment, a responsive measure can include prompting the subjectto open a vent and/or adjust a fan speed to increase air flow.

In an embodiment, a responsive measure can include prompting the subjectto maintain their gaze at a fixed point.

In an embodiment, a responsive measure can include prompting the subjectto maintain their gaze at a fixed point on the horizon no less than 30degrees different than the direction of a vector representing theircurrent motion.

In an embodiment, a responsive measure can include prompting the subjectto breath according to a predetermined cadence.

In an embodiment, a method further can include subclassifying avehicular classification based on at least one of the tracked motion,tracked head position, vibrations, ambient sound and magnetic field asone of: a passenger vehicle, a bus, a train, a plane, and a boat.

In an embodiment, a motion sickness prevention system is included havinga control circuit, a motion sensor in electrical communication with thecontrol circuit, wherein the motion sensor is disposed in a fixedposition relative to a head of a subject wearing the hearing assistancedevice, a microphone in electrical communication with the controlcircuit, an electroacoustic transducer for generating sound inelectrical communication with the control circuit, a power supplycircuit in electrical communication with the control circuit, whereinthe control circuit is configured to track motion of the subject usingthe motion sensor, estimate vestibular system input based on trackedmotion of the subject, track head position of the subject using themotion sensor, and estimate visual system input based on tracked headposition of the subject.

In an embodiment, a motion sickness prevention system is included havinga control circuit, a motion sensor in electrical communication with thecontrol circuit, wherein the motion sensor is disposed in a fixedposition relative to a head of a subject wearing the hearing assistancedevice, a microphone in electrical communication with the controlcircuit, an electroacoustic transducer for generating sound inelectrical communication with the control circuit, a power supplycircuit in electrical communication with the control circuit, whereinthe control circuit is configured to track motion of a subject using themotion sensor, track head position of a subject using the motion sensor,sense vibrations with a vibration sensor, sense ambient sounds with themicrophone, sense magnetic fields with a magnetic field sensor, andclassify the surroundings based on the tracked motion, tracked headposition, sensed vibrations, sensed ambient sound, and sensed magneticfields, as one of vehicular, stationary, and non-stationary, wherein thecontrol circuit in further configured to initiate a responsive measurebased on the classification of the surroundings.

It should be noted that, as used in this specification and the appendedclaims, the singular forms “a,” “an,” and “the” include plural referentsunless the content clearly dictates otherwise. It should also be notedthat the term “or” is generally employed in its sense including “and/or”unless the content clearly dictates otherwise.

It should also be noted that, as used in this specification and theappended claims, the phrase “configured” describes a system, apparatus,or other structure that is constructed or configured to perform aparticular task or adopt a particular configuration. The phrase“configured” can be used interchangeably with other similar phrases suchas arranged and configured, constructed and arranged, constructed,manufactured and arranged, and the like.

All publications and patent applications in this specification areindicative of the level of ordinary skill in the art to which thisinvention pertains. All publications and patent applications are hereinincorporated by reference to the same extent as if each individualpublication or patent application was specifically and individuallyindicated by reference.

The embodiments described herein are not intended to be exhaustive or tolimit the invention to the precise forms disclosed in the followingdetailed description. Rather, the embodiments are chosen and describedso that others skilled in the art can appreciate and understand theprinciples and practices. As such, aspects have been described withreference to various specific and preferred embodiments and techniques.However, it should be understood that many variations and modificationsmay be made while remaining within the spirit and scope herein.

1-49. (canceled)
 50. A motion sickness prevention system comprising: anear-wearable device comprising a control circuit; and a motion sensor inelectrical communication with the control circuit, wherein the motionsensor is disposed in a fixed position relative to a head of a subjectwearing the ear-wearable device; wherein the control circuit isconfigured to track motion of the subject using the motion sensor;estimate vestibular system input based on tracked motion of the subject;track head position of the subject using the motion sensor; estimatevisual system input based on tracked head position of the subject; andestimate consistency between the vestibular system input and the visualsystem input.
 51. The motion sickness prevention system of claim 50,wherein the control circuit is configured to initiate a responsivemeasure if the estimated consistency crosses a threshold value.
 52. Themotion sickness prevention system of claim 51, the responsive measurecomprising prompting the subject to move their head sufficiently toincrease consistency between the vestibular system input and the visualsystem input.
 53. The motion sickness prevention system of claim 51, theresponsive measure comprising providing auditory stimulation.
 54. Themotion sickness prevention system of claim 51, the responsive measurecomprising a perceptually stationary auditory stimulus.
 55. The motionsickness prevention system of claim 51, the responsive measurecomprising prompting the subject to focus their gaze on the horizon. 56.The motion sickness prevention system of claim 51, wherein the controlcircuit is configured to use tracked head position of the subject toassess whether the subject focused their gaze on the horizon.
 57. Themotion sickness prevention system of claim 51, wherein the controlcircuit is configured to issue instructions to an external device todisplay video images on the external device showing motion consistentwith tracked motion.
 58. The motion sickness prevention system of claim51, wherein the control circuit is configured to estimate visual systeminput based on tracked head position of the subject by estimating adirection of visual focus of the subject.
 59. The motion sicknessprevention system of claim 51, wherein the control circuit is configuredto estimate vestibular system input based on tracked motion of thesubject by estimating a motion vector.
 60. The motion sicknessprevention system of claim 51, wherein the control circuit is configuredto estimate vestibular system input based on tracked motion of thesubject and tracked head position of the subject.
 61. A motion sicknessprevention system comprising: an ear-wearable device comprising acontrol circuit; a motion sensor in electrical communication with thecontrol circuit, wherein the motion sensor is disposed in a fixedposition relative to a head of a subject wearing the ear-wearabledevice; a microphone in electrical communication with the controlcircuit; a magnetic field sensor in electrical communication with thecontrol circuit; wherein the control circuit is configured to perform atleast one of track motion of a subject using the motion sensor; trackhead position of a subject using the motion sensor; sense vibrationswith using the motion sensor; sense ambient sounds with the microphone;sense magnetic fields with a magnetic field sensor; and classify thesubject's surroundings based on at least one of tracked motion, trackedhead position, sensed vibrations, sensed ambient sound, and sensedmagnetic fields, as one of vehicular; stationary; and non-stationary;and initiate a responsive measure based on the classification of thesurroundings.
 62. The motion sickness prevention system of claim 61,wherein the control circuit is further configured to estimate vestibularsystem input based on tracked motion of the subject; track head positionof the subject using the motion sensor; estimate visual system inputbased on tracked head position of the subject; and estimate consistencybetween the vestibular system input and the visual system input.
 63. Themotion sickness prevention system of claim 62, wherein the controlcircuit is configured to initiate a responsive measure if the estimatedconsistency crosses a threshold value.
 64. The motion sicknessprevention system of claim 63, the responsive measure comprisingprompting the subject to move their head sufficiently to increaseconsistency between the vestibular system input and the visual systeminput.
 65. The motion sickness prevention system of claim 63, theresponsive measure comprising prompting the subject to maintain theirgaze at a fixed point.
 66. The motion sickness prevention system ofclaim 63, the responsive measure comprising at least one of providingauditory stimulation, providing a perceptually stationary auditorystimulus, prompting the subject to focus their gaze on the horizon, andprompting the subject to lift their head to a horizontal position if atracked head position is indicated to be below or above horizontal. 67.The motion sickness prevention system of claim 62, wherein the controlcircuit is further configured to query the subject regarding theirstatus if the estimated consistency crosses a threshold value.
 68. Themotion sickness prevention system of claim 61, wherein the controlcircuit is configured to use tracked head position of the subject toassess whether the subject focused their gaze on the horizon.
 69. Themotion sickness prevention system of claim 61, wherein the controlcircuit is further configured to subclassify a vehicular classificationbased on at least one of the tracked motion, tracked head position,vibrations, ambient sound and magnetic field as one of a passengervehicle, a bus, a train, a plane, and a boat.